Online Algorithms for Dynamic Matching Markets in Power Distribution Systems

This paper proposes online algorithms for dynamic matching markets in power distribution systems, which at any real-time operation instance decides about matching -- or delaying the supply of -- flexible loads with available renewable generation with the objective of maximizing the social welfare of the exchange in the system. More specifically, two online matching algorithms are proposed for the following generation-load scenarios: (i) when the mean of renewable generation is greater than the mean of the flexible load, and (ii) when the condition (i) is reversed. With the intuition that the performance of such algorithms degrades with increasing randomness of the supply and demand, two properties are proposed for assessing the performance of the algorithms. First property is convergence to optimality (CO) as the underlying randomness of renewable generation and customer loads goes to zero. The second property is deviation from optimality, is measured as a function of the standard deviation of the underlying randomness of renewable generation and customer loads. The algorithm proposed for the first scenario is shown to satisfy CO and a deviation from optimal that varies linearly with the variation in the standard deviation. But the same algorithm is shown to not satisfy CO for the second scenario. We then show that the algorithm proposed for the second scenario satisfies CO and a deviation from optimal that varies linearly with the variation in standard deviation plus an offset

On the Verification of Deep Reinforcement Learning Solution for Intelligent Operation of Distribution Grids

Capabilities of deep reinforcement learning (DRL) in obtaining fast decision policies in high dimensional and stochastic environments have led to its extensive use in operational research, including the operation of distribution grids with high penetration of distributed energy resources (DER). However, the feasibility and robustness of DRL solutions are not guaranteed for the system operator, and hence, those solutions may be of limited practical value. This paper proposes an analytical method to find feasibility ellipsoids that represent the range of multi-dimensional system states in which the DRL solution is guaranteed to be feasible. Empirical studies and stochastic sampling determine the ratio of the discovered to the actual feasible space as a function of the sample size. In addition, the performance of logarithmic, linear, and exponential penalization of infeasibility during the DRL training are studied and compared in order to reduce the number of infeasible solutions

Risk-based Stochastic Continuous-time Scheduling of Flexibility Reserve for Energy Storage Systems

This paper develops a novel risk-based stochastic continuous-time model for optimizing the role of energy storage (ES) systems in managing the financial risk imposed to power system operation by large-scale integration of uncertain renewable energy sources (RES). The proposed model is formulated as a two-stage continuous-time stochastic optimization problem, where the generation of generating units, charging and discharging power of ES, as well as flexibility reserve capacity from both resources are scheduled in the first stage, while the flexibility reserve is deployed in the second stage to offset the uncertainty of RES generation in each scenario. The Conditional Value at Risk (CVaR) is integrated as the risk metric measuring the average of the higher tail of the system operation costs. The proposed model is implemented on the IEEE Reliability Test System using load and solar power data of CAISO. Numerical results demonstrate that the proposed model enables the system operators to effectively utilize the flexibility of ES and generating units to minimize the system operation cost and renewable energy curtailment at a given risk tolerance level

Continuous-time Look-Ahead Scheduling of Energy Storage in Regulation Markets

Energy storage (ES) devices offer valuable flexibility services, including regulation reserve, in power systems operation that could improve the reliability and cost-efficiency of systems with high penetration of renewable energy resources. In this paper, a continuous-time look-ahead regulation capacity scheduling model is proposed, which more accurately models and schedules the regulation capacity trajectories provided by generating units and ES devices in real-time power systems operation. A function space solution method is proposed to reduce the dimensionality of the continuous-time problem by modeling the parameter and decision trajectories in a function space formed by Bernstein polynomials, which converts the continuous-time problem into a linear programming problem. Numerical results, conducted on the IEEE Reliability Test System, show lower operation cost and less regulation scarcity events in real-time power systems operation due to efficient deployment of the ES flexibility in regulation markets

Hierarchical Flexibility Offering Strategy for Integrated Hybrid Resources in Real-time Energy Markets

This paper proposes a hierarchical model for determining the energy flexibility offering strategy of integrated hybrid resources (IHRs) in power distribution systems to participate in real-time energy markets. The proposed model utilizes the scalability, fast response time, and uncertainty observation of deep reinforcement learning (DRL) to overcome the scalability issue of operating numerous flexible resources and deliverability of energy flexibility to the real-time markets in the presence of the network constraints. To that end, the power distribution system is divided into multiple IHRs, where different types of flexible loads, energy storage systems, and solar plants with controllable inverters are operated through local IHR controllers, trained by deep deterministic policy gradient (DDPG) algorithm. Active power request and reactive power capacity of IHRs are then transmitted to a central flexibility controller, where a quadratic optimization model ensures the deliverability of the energy flexibility to the real-time energy market by satisfying the distribution network constraints. The proposed model is implemented on the 123-bus test power distribution system, demonstrating the capability of DRL-based hierarchical model for scalable operation of IHRs in order to offer deliverable energy flexibility to the real-time energy market

Continuous Hydrothermal Flexibility Coordination Under Wind Power Uncertainty

This paper develops a stochastic continuous-time optimization model for coordinating the operation of flexibility in a hybrid hydro-thermal-wind power system. The developed model gives insight for investigating the short-term interactions between the different generation technologies. The continuous-time model captures the sub-hourly variations of wind power and load, and can accurately model the ramping capability of the system. A simplified Northern European system is studied over a 30 hour period to examine the potential of using hydropower as a comprehensive flexibility provider. Norwegian hydropower is shown to be a significant source of flexibility used to mitigate wind power variations, especially during ramping constrained periods. The hydropower provides 73.5% of the balancing energy in the base case, which includes smoothing out longer wind power deviations as well as rapid ramping relief. The short-term implications of increasing the offshore wind power in the North Sea by 50% compared to 2020 was also studied in the Northern European test system. The increased wind power causes steeper ramping in the net load, which drives the hydropower to its full balancing potential to allow thermal units to operate within their ramping limits.Continuous Hydrothermal Flexibility Coordination Under Wind Power UncertaintyacceptedVersio

Scalable Grid-Aware Dynamic Matching using Deep Reinforcement Learning

This paper proposes a two-level hierarchical matching framework for Integrated Hybrid Resources (IHRs) with grid constraints. An IHR is a collection of Renewable Energy Sources (RES) and flexible customers within a certain power system zone, endowed with an agent to match. The key idea is to pick the IHR zones so that the power loss effects within the IHRs can be neglected. This simplifies the overall matching problem into independent IHR-level matching problems and an upper-level optimal power flow problem to meet the IHR-level upstream flow requirements while respecting the grid constraints. Within each IHR, the agent employs a scalable Deep Reinforcement Learning algorithm to identify matching solutions such that the customer's service constraints are met. The central agent then solves an optimal power flow problem with the IHRs as the nodes, with their active power flow and reactive power {capacities}, and grid constraints to scalably determine the final flows such that matched power can be delivered to the extent the grid constraints are satisfied. The proposed framework is implemented on a test power distribution system, and multiple case studies are presented to substantiate the welfare efficiency of the proposed solution and the satisfaction of the grid and customers' servicing constraints